Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Print assertion phase when verifying rule result #52

Merged
merged 1 commit into from
Nov 1, 2024
Merged

Print assertion phase when verifying rule result #52

merged 1 commit into from
Nov 1, 2024

Conversation

yuumasato
Copy link
Member

When a rule's result doesn't match the expected result, let's also print the phase in which it happened, whether the default result or after remediation is applied.

@yuumasato
Print assertion phase when verifying rule result
cb2f055 
When a rule's result doesn't match the expected result, let's also print
the phase in which it happened, whether the default result or after
remediation is applied.
@yuumasato
Copy link
Member Author

This is just a small quality of life improvement.

When I'm checking out the e2e results I search for to the rule didn't match lines and consistently find myself scrolling up or down to figure out whether the scan I'm looking at is the before or after remediations...

@yuumasato
Copy link
Member Author

The moderate test failed, but that showcased the patch in action.

medium
    helpers.go:879: E2E-FAILURE: The expected remediated result for the e2e-moderate-resource-requests-limits-in-deployment rule didn't match. Expected 'PASS', Got 'FAIL'
    helpers.go:872: Result - Name: e2e-moderate-resource-requests-limits-in-statefulset - Status: FAIL - Severity: medium
    helpers.go:879: E2E-FAILURE: The expected remediated result for the e2e-moderate-resource-requests-limits-in-statefulset rule didn't match. Expected 'PASS', Got 'FAIL'

@yuumasato
Copy link
Member Author

Another example, here it is not clear whether the issue is on the default result or after remediation.
And the issue is actually with after_remediation result.

     helpers.go:872: Result - Name: e2e-pci-dss-node-4-0-worker-file-permissions-cni-conf - Status: PASS - Severity: medium
    helpers.go:879: E2E-FAILURE: The expected result for the e2e-pci-dss-node-4-0-worker-file-permissions-cni-conf rule didn't match. Expected 'FAIL', Got 'PASS'

rhmdnd
rhmdnd approved these changes Sep 30, 2024
View reviewed changes
Copy link
Collaborator

@rhmdnd rhmdnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@yuumasato
Copy link
Member Author

/test e2e-aws-ocp4-moderate

@yuumasato
Copy link
Member Author

/retest-required

@yuumasato yuumasato merged commit 4120313 into ComplianceAsCode:main Nov 1, 2024
4 checks passed
@yuumasato yuumasato deleted the print-phase-of-failure branch November 1, 2024 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rhmdnd rhmdnd rhmdnd approved these changes

@JAORMX JAORMX JAORMX approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@yuumasato @JAORMX @rhmdnd